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THE PERSISTENCE OF A VISUAL DOMINANCE EFFECT 
IN A TELE MANIPULATION TASK : 
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In the two experiments reported, we have examined the effi- 
ciency of an electrotactile stimulation for manipulation 
control. These studies were based on a psychological point 
of view and issued from the ergonomical purpose to feedback 
information to a severely handicaoped operator, such as a 
quadriplegic, to control a telemanipulator. We have genera- 
lised the telemanipulation situation to non handicapped 
people operating with a hand controller and compared visual 
to electrotactile feedback. This sensory feedback informed 
operators on the sum forces exerted by the terminal device 
on the environment. An experimental material made of an 
orientable slider on a stem was developped which allowed 
to teat the control of the three cartesian coordinates 
and the forces involved by the manipulator during a linear 
constraining movement task. Different cognitive activities 
were hypothesised in three situations : a first one where 
operators performed the task in a viewing situation, a 
second one where they were in a blind-folded situation but 
knew the spatial orientation of the stem and a third one 
where they had to operate blind-folded and guess the spatial 
orientation of the stem. Attention demand, type and quantity 
of information feedback were experimentaly modified so as 
to examine different effects such as visual dominance, 
alerting effect and treatment capacity. Main results showed 
chat an electrotactile frequency modulation (1-50 Hx.) 
provided information which is treated as an on/off signal 
and is longer to access cognitive treatment and to develop 
operative image than visual information. Comparatively 
equivalent visual information is more sensible and makes 
decision movement faster, showing a permanent visual 
dominance effect even if the operator's attention is centered 
on the tactile modality. 



In teleoanipulation control evidence has been established that a force 
feedback system improves operator performance (Hill, 1976, Vertut and al. 
1976). In the CEA - La CalhSne KA 23 master-slave manipulator system the 
force feedback is delivred as a force reflecting the force vector on the 
slave handled by an operator. The Psychological interpretation for this 
improvement is quite simple if we assume that the for^e feedback presents 
an information increasing the operator's knowledge of the results of his 
control actions. Therefore movement correction and initiation can be 
ia^roved by increasing such feedback to the operator. However if no force- 
reflecting master-slave manipulation is used, in what other may could the 
force information be presented ? This question is posed in particular when 
the master is replaced by a "syntaxer", a control transducer combining 
6 degrees of freedom in a single handle. 

As one possibility we studied the use of an electrotactile stimulation to 
return information to the operator and to test the efficiency of the 
electrotactile modality in teleoperator control. Several studies have been 
published on electrotactile stimulation, showing that a frequency modula- 
tion may be used (Shannon, 1976; Szeto, Prior and Lyman, 1977; Tachi, 

Tanie and Abe, 1978; Kato and al. 1970; Solomonow, Lyman and Freedy, 1977: 
Solomonoif end Lyman, 1978). However, thus for it is not clear what 
cognitive processes are involved in presenting the information provided 
by different sensory modalities. One could argue that che electrical 
stimulation is transmitted to the cognitive center faster than visual or 
auditory stimulus. However, the attention required to code and stock 
information may depend on the modality through which this information 
accesses the cognitive treatment centers and whether parallel processing of 
information is possible when different modalities are stimulated simulta- 
neously. 

In Hill's studies (1974) no clear results have been obtained with a couch 
feedback system in telemanipulation tasks as compared to a visual feed- 
back and he concluded that Che learning effect was to strong to observe 
an effect on the type of display. The author recommended to alternate 
experimental variables in order Co reduce the variance caused by learning. 

Earlier Bliss and al. demonstrated that a tactile display augmented the 
human response time in comparison to the one for a visual display. 

However, their stimulation was mechanical and for electrical stimulation 
the results might be different. 

In manual manipulation visual and kinesthetic information is used to 
control movements, information is delivered blmodaly and attention is 
divided between two sensory modalities. Some results have shown (Treisman 
and Davies, 1973) Chat dividing attention between two different modalities 
is more efficient Chan dividing attention between signals of the same 
modality. Ue hypothesized Chat in Celeoperator control where vision is 
focused on controlling the task, force feedback on a visual display 
presents a situation where attention is divided on the same modality 
(vision). In comparison, force feedback to the operator through the 
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tactile modality divides his attention between two different modalities 
and could be more efficient. 

This result is explained (Richard, 1980) by assuming that the limitations 
of divided attention are in part caused by the conditions of the 
reception and storage of the sensory input. 

Basically, two different models of information processing were diseased. 
In the Broadbendt's model semantic information is treated after its 
selection and is limited in capacity or sequencially processed. In the 
Deutsch and Deutsch's model sie.mantic analysis is processed before the 
selection of information which could be treated in parallel. Various 
intermediate models have been proposed since this two first one (See 
Richard, I960 for this question). As a model we retained the hypothesis 
that a first kind of semantic analysis could be effectuated in a sensory 
memory register before transfer to the short-term-memory. Final treatment 
could be of a sequential nature, while surface features of the stimulus, 
such as physical characteristics and some semantic features, could be 
processed in parallel, especially in the case of perception through 
multiple sensory channels. If this is the case, decision and motor res- 
ponses should be better in a bimodal situation than in a monomodal one. 

Another argument concerns the electrical skin stimulation we used. 

Stevens (1938) noted that "the electrical stimulus has its principal 
effect directly on the sensory nerve fibers. That is, electrical stimu- 
lation bypasses the receptive structure". So if a first treatment of 
the information is effectuated in the receptive structure before its 
transfer to short-term-memory, electrical stimulation bypassing this 
first structure will directly access short- term-memory without a first 
treatment. The consequence will be a loss of information if this 
peripheral structure is able to initiate a first treatment of the 
information (see diagram I). 


visual 

stimulation 

mechanical 
stimulation 
of the skin 

electrical 
stimulation 
of the skin 



Peripheral structures 


Central structure 


Diagram I ; Hypothetical Peripheral Structure for visual stimulation, 
mechanical and electrical stimulation of the skin. 
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The transfer function could initiate a first treatment of the information. 

In the case of the visual stimulation information could be stored for a 
short time in a sensory memory register where information could be retrieved 
(Sperling, 1980). 

In a previous experiment (Gaillard, 1981) testing the effect of an 
electrical feedback in a telemanipulation task, we did not find a signi- 
ficant effect on the execution time of the task. However, the number of 
errors with electrotactile feedback was greater than with an equivalent 
visual feedback. We concluded of that the gain in performance when dividing 
attention between two modalities could be null with electrical stimulation. 
Therefore, we have developped two experiments without direct vision of the 
task to study in more detail the role of the peripheral structure as a 
function of the modality of the feedback and their effects on motor control. 

EXPERIMENT 1 


METHOD 


- Subjects : Two subjects were trained in telemanipulation control. They 
had been operators in previous experiments and work in the SPARTACUS 
Laboratory. 

- Apparatus : The experimental system was composed of a manipulator type 
CEA La CalhSne MA 23 with six degrees of freedom plus gripper (Photo I). 

The manipulator was controlled by a "syntaxer" with six simultaneously 
controllable degrees of freedom. However only three of them were used 
for this experiijent, controlling the three translations in the Cartesian 
coordinates. These three translations were operated with a velocity 
control by three micro-displacements of the handle. These micro-displa- 
cements required a certain amount of fcrce to be exerted on the syntaxer 
(Photo 2) . 

Manipulator and syntaxer were connected through a SOLAR 16/65 computer, 
programmed to allow the operator to control the manipulaLor using a 
specific language linking the three degrees of freedom of the syntaxer 
to the three translations of the gripper in space. Initially, the system 
and its program were studied for a medical manipulator as a part of the 
SPARTACUS project as an aid to high-level quadriplegics (Guittet and al. 
1979). Then the same concepts were used to control an industrial manipu- 
lator, but through another control transducer and an adapted control 
language . 

Two types of sensory feedback were used to inform the operator about the 
forces exerted by the gripper on the axis. The first one was ts visual 
display of 5 LED's with each diode successively representing a higher 
level of force. One diode lit meant the sum of the forces was low, but 
not negligible, and S diodes lit that the gripper was exerting a strong 
force. The sum of the forces was obtained from the summation of the 
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currents of the torque motors approximately corresponding to the three • 
cartesian coordinates . 

The second display was an electrotactile stimulation situated on the 
operator's arm with the following characteristics : 

•• rectangular pulse train 

- frequency modulation 1 -»• 50 Hz 

- pulse duration 150 ps 

- pulse intensity 5 mA. 

These characteristics allowed the operator to distinguish 5 levels of 
frequency and was sufficiently confortable. (Kato and al. 1970, Szeto, 

Prior and Lyman, 1977). The frequencies varied proportionaly with the 
sum of the forces. 

- Task . The task consisted of the displacement of a cursor along an 
orientable axis chat could be rigidly fixed with chree different incli” 
nations and twelve horizontal orientations (Photo 3) . The cursor was linked 
to the axis by a self-aligning bearing to avoid variation of sliding 
friction. The task required the operator to precisely control manipulator 
movements in the space and to guide the cursor held by the gripper into a 
direction strictly parallel with the orientation of the axis. 

The operator was in a "blind folded" situation. If the gripper was not 
guided in parallel with the axis, an increasing effort was generated and 
the cursor could not be conducted to the limit switch it the end stop. 

The sliding distance remained the same all over the experiment. The 
execution time was measured automatically at each trial. When the cursor 
began to move a microswitch was released, starting a chronometer and 
when the cursor had moved about 40 cms on the axis, a record microswitch 
was activited stopping the chronometer. 

Another way to finish a trial was by the triggering of an automatic safety 
stop when sum of the tr *..es exerted a certain level . Then the syntaxev 
was automatically uncr .,/ied to the manipulator and the trial was considered 
as a failure. 

- Design and Procedure . Three conditions of sensory feedback were permuted 
on 30 trials, 10 trials were run with visual feedback, 10 with electro- 
tactile feedback and 10 without force feedback. The experimental design 
was X X T|q with S for subjects, M for sensory modality and T for 

trials. Of the 30 trials, 21 required the control of three degrees of 
freedom and 9 required only two degrees of freedom. The orientation of the 
axis was changed for each trial and there were never two identical orien- 
tations in the 30 trials. Before each trial the experimenter fixed the 
axis with a new orientation and p«. litlomd the gripper on the cursor at 
the starting position . The operator saw the orientation of the axis and 
memorized it. Then, he turned h\s back to the axis, grasped the syntaxer, 
and started the trial controlling the movement only using the sensory 
feedback (visual or electrotactile) in 20 trials out of 30 and without 
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sensory feedback, only relying on his memory of orientation, in 10 trials 
out of 30. After each trial he was informed on his execution time of the 
task. 

- Experimental predictions . Under electrotactile feedback condition we 
expected the information, bypassing the receptive structure, to be entered 
directly into short-term-memrry (STM) where its treatment requires more 
time to be correctly int ^rpreted than under the visual feedback condition 
where a first treatment is efiifCtuated before its access to STM. Memory 
load in STM will be greater under the electrctar.tile condition than under 
the visual one and task time will be longer in the former case. 

RESULTS 


Task time for the 2 degrees of freedom trials were not significantly 
different F(I,2) « 1,27; p > 10 whereas they were different for the 
3 degrees of freedom trials F(I,6) - 8,68; p < .05. No other factors, nor 
interaction were significantly different. Average times and nunber of 
failures were as listed in table 1 . 



task time 
for 3 df 

failure 

task time 
for 2 df 

failure 

visual 

13,06 

2 

12 

0 

electrotactile 

17,13 

0 

19,93 

1 

no feedback 

11,53 

1 1 

14,35 

3 


Table 1 - Taak time in eeeonda and number of failures as 
a function of sensory feedback 


DISCUSSION 


As predicted the average time was longer with an electrotactile feedback, 
indicating that the task time is varying with the information treatment 
and memory capacities required and not <^ith its transmission delay from 
peripheral to central structures. The number of failures as a function 
of sensory feedback showed that force feedback was of a great use to 
complete che task and even made it possible to execute it without feed- 
back, only using the memorised spatial orientation of the axis. The 
information presented by electrotactile feedback was effectively treated 
and interpreted as indicated by a low number of ft.ilures and a longer 
task time. It suggested that electrotactile infotioation was treated in 
STM and that the limit of the capacity of treatment and retention was 
not reached. The results also suggested that visual information could 
receive a first treatment in the peripheral structure before its access 
to STM, thereby facilitating this subsequent treatment. In this 
experiment feedback information had to be interpreted a:-d compared with 
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the information of the memorised orientation of the axis and physical 
analysis of the stimulus could be sufficient to perform this comparison 
in memory. Now a question was raised about a semantic interpretation of 
electrotactile information. 


EXPERIMENT 2 


METHOD 


- Subjects, apparatus and task remained the same as in the first experiment 
except for the blindfold task that had to be completed without a prior 
knowledge of the orientation of the axis. The subjects had to estimate 

the orientation of the axis during manipulation. In one half of the trials 
an additional task had to be completed simultaneously during the manipu- 
lation task. In this additional task, or semantic task, the operator had 
to listen to a text and had to press a button when he identified a proper 
name in the text. Subjects, therefore, were in a condition of divided 
attention. 

- Procedure . There were four different conditions, two with attention 
focused on motor control with visual or electrotactile force feedback, 
and two with attention divided between the semantic task and the motor 
control with visual or electrotactile feedback. There were 40 trials for 
each subject, 10 for each condition. Task time, number of failures and 
responses in the semantic task were recorded. In addition, we recorded 

in X, Y recorder the sum of the forces exerted by the gripper during each 
trial and we asked the subject his estimation of the orientation of the 
axis at the end of the trial. 

- Experimental predictions . To complete the telemanipulation task the 
subject interpreted the feedback as a confirmation or rejection of an 
estimated orientation of the axis, and could build a cognitive represen- 
tation of the axis in space. Assuming that the electrotactile stimulation 
bypassed the receptive structures, we predicted that in the focused 
attention condition task time and failure rate would be more inq)ortant 
with an electrotactile feedback than with a visual display as in 
experiment I . If we hypothesize that a first treatment of the semantic 
information is made before its input in STM , then the performance will 
not be dramatically affected in divided attention. However, if a first 
treatment of semantic information is only made in STM, performance 
should be affected by' this additional task. In this case it means that 
semantic information bypasses the sensory register. Indeed, we hypothesized 
a conscious process to interprete the sensory feedback to build an operative 
image of the axis in space and that process should at least involve STM 
storage. In other words, if semantic information is selected before its 
access in STM, in a preconscious activity, interpretation of the stimulus 
feedback should be made. If semantic information is selected in a conscious 
working memory, interpretation and operative image formation should be 
difficult. In the latter case, task time and failure rate should increase 
and the number of motor response decrease. 
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RESULTS 


Mean task time and number of failures were calculated for each sensory 
conditions and were the following : Table 2. 


Sensory 

conditions 

Visual 

Electrotactile 

Visual 

divided 

Electrotactile 

divided 

Dependent 

variables 

Task 

time 

Failures 

Task 

time 

Failures 

Task 

time 

Failures 

Task 

time 

Failures 

V' 

54.2 

1 

86,2 

22 

98,1 

9 

115,7 

16 

^ s„ 

- 

70.1 

2 

93,5 

4 

87,5 

4 

93,5 

8 

m 

62.1 

1.5 

89,8 

13 

92,8 

6.5 

104,6 

12 


Table 2 - Mean task time in seconds and number of failures 

For the additional semantic task the percentage of alarm was 95% with visual 
feedback in the motor task and 96 % with electrotactile feedback. The 
numbers of false alarms were, respectively, 19 or 20. Guessing of the 
orientation of the axis was calculated from the number of degrees of 
freedom correctly estimated by the operator after each trial. No significant 
differences were found between visual and electrotactile feedback (for 
the two operators were, respectively, 0.75 and 0.79 p > .10). The 
proportions of degrees of freedom correctly estimated were 72 % for the 
first operator and 54 % for the second one. Ranking the execution times 
in the condition of focused attention indicates a significant difference 
between visual and electrotactile (Sign test N ■ 20, x ■ 1; p < .01). 

The difference was not significant in divided attention (sign test N ■ 20, 

X ■ 9; p > .10). The only factor approaching the level of .10 in the 
variance analyse was the sensory condition F(3,27) ■ 2,26. In return v^e 
found a large difference between the two sensory feedbacks for the nuiaber 
of motor responses t(38) ■ 7,82; p < .001. The rate of motor response was 
calculated by dividing the number of responses by the task time for each 
trial and is indicated in table 2. 


Visual 

Electrotactile 

Visual 

divided 

Electrotactile 

divided 

0,68 

0,38 

0,66 

0,31 


Table 2 - Bate of motor response 
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DISCUSSION 


As expected results corresponded with those obtained in the first 
experiment. With electrotactile feedback both tajk execution time and 
number of failures increased. The score obtained in the semantic task 
indicated a good semantic detection when attention was divided on two 
different sensory channels. But the effects on task time and number of 
failures correspond with a decrease of performance in the telemanipulation 
task. Guer Lng the spatial orientation of the axis was more difficult with 
an electrotactile feedback and with attention divided, but the formation 
of a consciour operative image of the axis in space was necessary to 
complete the task and this mental image obtained with both sensory feed- 
backs under all c< editions, as suggested by the number of degrees of 
freedom guessed after each trial completed. 

More interesting is the number of motor responses obtained under respec- 
tively, visual and electrotactile* feedback. Dividing attention had no 
effect on either the number of motor responses or on motor decision. 

Task time was increased with electrotactile because the operator could 
not make motoi decisions during part of the time. This result suggests 
that the interpretation of the electrotactile feedback is more difficult 
and not as sensitive as of an equivalent visual feedback, and supports 
the general assumption that an electrotactile stimulation bypasses the 
peripheral structure. If so, physical features of t .e stimulus are not 
treated in this first stage and have to be treated in a working memory 
where the stimulus is roughly interpreted. Results also indicated that 
the performance in motor control and formation of a compatible operative 
image the axis with a visual feedback is more affected by the additional 
semantic task. It could be hypothesized that the semantic information is 
processed in a working memory after a preliminary treatment of the physical 
features of the stimulus in a sensory memory register. This suggests that 
the semantic treatment is not effectuated in parallel, but sequentially 
and that it is limited in capacity. With the electrotactile feedback, 
where no motor decision could occur for relative long periods of time, 
semantic information can be processed and does not interfere with the 
motor control . 


CONCLUSION 


The two experiments reported here were not designed to generalize the 
results to the population. Generalizations have been made for the 
repetition of the tasks and do only suggest direction for further research 
in the field of general psychology. The aim of this lesearch was to 
investigate the possibility to use an electrotactile stimulation in 
teleoperation and to observe the interpretation of such information as a 
feedback to the operators. It leads us to formulate cognitive hypothesis 
and to propose the following assumptions : 

I. A visual feedback is more informative than an electrotactile one, 
suggested by the general visual dominance effect. 



2. A complex electrotactile feedback : 

- slows down both the motor decision and motor response processes. 

- is processed as an all or nothing signal. 

- bypasses the receptive structure and accesses directly in a working 
memory where information is to be sequentially processed and where memory 
is limited in treatment capacity* 

Finally, the electrotactile stimulation could be used more as an alerting 
signal than a complex information feedback. It does not seem to be well 
adapted to code semantic or mental image information. Visual or auditory 
information, however, does benefit from the transfer function in the 
peripheral structure where information could be treated before its access 
to a working memory. The visual dominance effect could be the result of 
the advantage of both a transfer function and a sensory memory register 
where information is pretreated and memorized for a short time. Dividing 
attention has an effect on the acquisition of the information bvt not on 
the subsequent decision processes, suggesting that in motor tasks the 
attention variable is primarily affecting early cognitive activities. 
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